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ABSTRACT 



The Intervent ion/ Prevent ion Program in the Wake County 
Public Schools (North Carolina) provides support to students who are 
experiencing academic, behavioral, family, and emotional difficulties that 
interfere with learning. The Student Support Team (SST) at each school 
provides this support. The SST is a collaborative, multidisciplinary team 
that is responsible for developing an action plan that supports the 
individual student. During the 1997-98 school year, 3,963 students were 
referred to the SST program. Most were being served in the regular classroom. 
About 50% of those referred identified themselves as male. About 66% of the 
academic concerns of these students were reported as "somewhat" or "a lot" 
improved during the school year. Behavior concerns of referred students 
improved at all grade levels, from 54% for grades 6 through 8 to 79% for high 
school students. Strategies were categorized as classroom-based, school 
support-based, or community-based. For each strategy recommended and 
implemented, a higher percentage rated the effectiveness as "good" compared 
to "poor." The SST program appears to provide an important service for 
at-risk students. (Contains 34 figures.) (SLD) 
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Since 1994, the North Carolina Department of Public Instruction has 
provided grant funding for an Intervention/Prevention Program (I/P) in 
the Wake County Public School System. The program's mission is to 
provide support to students and youth who are experiencing academic, 
behavioral, family, and emotional difficulties which interfere with 
learning. The Student Support Team (SST) at each school provides 
this support. 

The SST is a collaborative, multidisciplinary team that is responsible 
for developing an action plan that supports and strengthens an 
individual student through the use of school and/or community-based 
strategies. 

The purpose of this evaluation was to respond to several questions 
related to the types of student referrals, the types of strategies 
recommended and implemented, and the effectiveness of the SST 
program. 

During the 1997-98 school year, a total of 3,963 students were 
referred. Most students referred to the SST did not have an 
exceptional student classification and were being served in the regular 
classroom. Approximately 50 percent of those referred were male and 
33 percent were female. The remaining 17 percent did not have sex 
coded on the scannable form. 

The two most prevalent areas of concern were academics and 
behaviors. Approximately 66 percent of academic concerns were 
reported as “somewhat” or “a lot” improved during the 1997-98 
school year. Behavior concerns improved at all grade levels. 
Approximately 64 percent of grades K-2, 65 percent of grades 3-5, 54 
percent of grades 6-8 and 79 percent of high school behavior concerns 
improved. 

Strategies were categorized as classroom-based, school support-based, 
or community-based. For each strategy recommended and 
implemented, a higher percentage rated the effectiveness as “good” 
compared to “popr.” 

The SST program provides an important service to students who are 
at-risk and their families. In most cases, referrals are handled in a 
timely manner. In addition, the students’ presenting concerns 
improved during the school year. 
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INTERVENTION/PREVENTION PROGRAM EVALUATION 

1996-1998 

In 1994-95, the Wake County Public School System received a grant from the North 
Carolina Department of Public Instruction to implement a pilot project known as the 
Intervention Prevention Program (I/P). The main purpose was to provide better support to 
students who were at-risk by collaborating with many human service agencies that work with 
children and families. The vehicle for change was the development of Student Support 
Teams (SSTs) to support students as a replacement for existing “teacher assistance te ams ” in 
nine schools. For the 1994-95 school year, a district/community committee selected schools 
through a competitive process for program participation. 

The SST is a collaborative, multidisciplinary team that is responsible for developing an 
action plan that supports and strengthens an individual student through the use of school 
and/or community-based strategies. The SST model involves viewing students from a 
holistic perspective. That is, the SST must be inquisitive about the dynamics of all aspects in 
the student’s environment and life circumstances to design appropriate and effective ways to 
increase success (e.g., academics, family, friends, affect, culture, behavior). The 
Coordinating Committee of the I/P program wrote and developed the Guide to Strong SSTs 
during the 1994-95 school year. Guide to Strong SSTs provided the framework for 
developing and implementing a successful SST process at an elementary, middle, or high 
school by defining specific goals for students and developing a plan of action to meet those 
goals. 

Since 1994, the North Carolina Department of Public Instruction has continued to provide 
grant funding that supports the Intervention/Prevention Program in the Wake County Public 
School System. The program's mission is to provide support to students and youth who are 
experiencing academic, behavioral, family, and emotional difficulties which interfere with 
learning. The SST at each school provided this support through a collaborative team 
approach, generating individualized strategies that influence the interaction of environment, 
family, emotions, and learning on the child’s life. The project grew, in three years, from the 
original group of nine schools piloting the project in 1994 (six elementary schools and three 
alternative schools), to 102 schools (67 elementary, 20 middle, 13 high, and 2 alternative 
school sites) during the 1997-98 school year. 

For the past two years, the Evaluation and Research Department collected and analyzed data. 
This summative evaluation of grant activities focused on measuring the impact of the SST 
interventions and services on the students and youth served. A systematic process of data 
collection generated data to evaluate student performance, areas of concern, and strategy 
effectiveness. 
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PROGRAM DESCRIPTION 

Student Support Teams are multidisciplinary (multi-agency) school-based teams that design 
and implement individualized action plans to strengthen and support students through school- 
based and/or conununity-based strategies. Implementation of the SST model consists of 
extensive training in three guiding practices. 

Individualized Interventions: generating strategies which influence the interaction of 
holistic influences on the child’s life (e.g., environment, family, emotional, learning). 

Collaborative Team Approach: involving parents, school staff, and community agency 
representatives in partnership to provide services needed by the child. 

Capitalizing on Strengths: remediating student weaknesses by building on strengths. 

The SST approach is designed to assist schools in helping students-at-risk. Many of these 
students are challenged with a variety of concerns at school and at home. These challenges 
include health, education, and social service needs, which may negatively affect the learning 
process. 

In public schools, a variety of factors associated with school failure may affect students at 
risk. Examples of risk factor conditions external to the school include poverty, unmet health 
needs and physical, emotional, and/or sexual abuse. Neither a single school nor a human 
service agency alone is capable of addressing the plethora of challenges these students face. 
In addition, students and families may often have difficulty in accessing school and 
community services. Consequently, students and families with numerous concerns may not 
know where or how to seek assistance. 

The school based SST builds and renews connections and partnerships among school and 
community based human service agencies to provide help wiA risk factors. Several essential 
functions portray SSTs at the school site. These include assessment, development of an 
action plan, linking students to services, implementation, monitoring, and evaluation. 
Through collaborative interactions, SSTs establish a comprehensive service network where 
participants share the responsibility for students at risk of school failure and their families. 
Therefore, students and families receive a variety of services in a supportive, orderly, and 
coordinated fashion. 
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EVALUATION PLAN 

Major issues addressed in the 1996-97 and 1997-98 evaluations focused on the impact of the 
SST process and the identification of interventions which appeared to be more successful 
than others in improving presenting concerns. The evaluation design also addressed specific 
questions about the SST program. For example, data was gathered indicating the time of 
initial referral and the initial meeting date. 

Evaluation Questions 

The purpose of this evaluation was to respond to several questions related to the types of 
student referrals, the types of strategies recommended and implemented, and the 
effectiveness of the SST program. 

□ Did the number of meetings and referrals occur at the same rate during the year? 

□ Were a large majority of the students identified as exceptional learners? 

□ Were presenting concerns more prevalent in any particular area(s)? 

□ Did students’ academic grades improve after SST interventions? 

□ Did students who were referred for behavior concerns show improvement after SST 
interventions? 

□ Did students who were referred for emotional stability concerns show improvement after 
SST interventions? 

□ Did students who were referred for “other” concerns show improvement after SST 
interventions? 

□ Were some strategies more effective than others? 

Data Collection Activities 

The Wake County School System's Department of Evaluation and Research collected and 
analyzed data for the 1996-97 and 1997-98 school years. Activities focused on measuring 
the impact of the SST interventions and services on the students served. A systematic 
process of data collection generated data to evaluate student performance, areas of concern, 
and strategy effectiveness. 

One technology innovation used for the evaluation was the creation of scannable program 
documentation. SST chairpersons completed scan forms on students referred to the Student 
Support Team. Compilation of the data from referral forms into a database allowed 
evaluators to provide pre-slugged information back to the SST coordinators who then 
provided comparison data and evaluated the effectiveness of their interventions. Educational 
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interventions encompassed areas of classroom-based strategies, school social work services, 
psychological services, family support services, school counseling services. Department of 
Health Services, community-based services referrals, special education referrals, and school- 
based support services. For 1997-98, strategies were grouped into three major 
classifications: classroom-based, school-based, and community-based. For both school 
years, SST coordinators collected baseline data when the SST discussed a student. A scan 
form was completed at the time of referral for each student. In June of each school year, the 
SST coordinators collected comparison data. 

SST coordinators were expected to collect the forms and verify that the information was 
correct. The data forms were submitted to the Department of Evaluation and Research for 
processing and data analysis. The culmination of the data analysis is this report. 
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EVALUATION RESULTS 

The Intervention/Prevention evaluation provides information related to demographics, 
presenting concerns, strategies and effectiveness, and changes in grades from referral to the 
end of the intervention. The demographic information provided includes gender and ethnic 
categories and student classification for special programs. 

Ethnic and Gender Characteristics of Referrals 

Of the 46 schools participating in the SST evaluation in 1996-97, there were 12 elementary 
schools, 19 middle schools, 12 high schools, and 3 alternative schools. SSTs completed scan 
forms for 1,279 students referred in 1996-97. Nearly twice as many males were referred as 
females. 



Figure 1 

Characteristics of SST Referrals for 1996-97 



Gender 


Black 


White 


Other 


Male (n=831) 


41% 


55% 


4% 


Female (n=448) 


48% 


50% 


2% 


Total (n=1279) 


43% 


53% 


3% 



During the 1997-98 school year, there were 67 elementary schools, 20 middle schools, 13 
high schools, and 2 alternative schools participating in the SST evaluation. Forms were 
completed for a total of 3,963 students. Of those referred, approximately 33 percent were 
female and 50 percent were male. For approximately 18 percent of student referrals, this 
data was unavailable. 



Figure 2 

Characteristics of SST Referrals for 1997-98 





Black 


White 


Other 


Male (n=1960) 


42% 


53% 


5% 


Female (n=1301) 


46% 


48% 


5% 


Total (n=3261) 


44% 


51% 


5% 
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Student Exceptionality Classification 

The largest group of students referred to the SST attended regular classes and did not receive 
special education or special services. This was true for the 1996-97 school year (83 percent 
attended regular classes and did not receive special services) and the 1997-98 school year (84 
percent). For the group of students who were receiving special education, specific learning 
disabled students received more referrals to SST (6 percent) than other groups in 1996-97. 
For the 1997-98 school year, students who were identified as speech/language impaired made 
up the largest number of special education services. Figure 3 summarizes student 
classification with percentages rounded. 



Figure 3 

Student Classification 



Student Classification 


1996-97 


1997-98 


None 


83% 


84% 


Behavior/ Emotionally Handicapped 


3% 


2% 


Gifted 


2% 


2% 


Hearing/ Vision Impaired 


0% 


0% 


Mentally Handicapped 


1% 


0% 


Specific Learning Disabled 


6% 


3% 


Speech/ Language Impaired 


2% 


4% 


Other 


3% 


5% 


Potential Dropout 


Not available 


2% 


ESL 


Not available 


2% 



A limited English proficient student is any student whose primary language is other than 
English and who is insufficiently proficient in the English language to receive instruction 
exclusively from regular educational programs and to function on an academic par with 
his/her peers*. The ESL or English as a Second Language Program is designed to help 
students who are limited English proficient to acquire the English language skills necessary 
to be successful in the classroom. The primary language of instruction is E n glish 

During the 1996-97 school year, information regarding ESL students was not collected. 
During the 1997-98 school year, two percent of the students who were referred to SST were 
participating in an ESL program. 



' See Guidelines for Testing Students with Limited English Proficiency: North Carolina Statewide Testing 
Program Grades 3-12. 
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Initial SST Request and First SST Meeting Dates 

During or immediately after the first SST meeting following the initial referral, a scannable 
form was to be completed for each student. Part of the data collection on the scannable form 
included information on the date of request for involvement (initial referral) and the date of 
the first SST meeting. This information provided a snapshot view of how long it took to 
initially plan for a referred student. Data was not collected on the date of implementation of 
suggested strategies. 

Figure 4 shows the initial request and date of the first meeting for the 1996-97 school year. 
After a small lag in the first two months of school, when referrals exceeded initial meetings, 
meetings and referrals occurred at about the same rate during the year with a decline in 
referrals toward the end of the year. Therefore, the SST waiting list appeared manageable. 

Figure 5 shows the initial request and date of the first meeting for the 1997-98 school year. 
Compared to the 1996-97 school year, there was more time between the initial referrals and 
first meetings during the early months of the school year. During both years, more students 
were referred and discussed in September, October, and November than during other periods 
of the school year. 

It should be noted that the number of referrals and initial meetings for the months of May, 
June, and July were very small. Lags and reasons for lags in meetings are not collected for 
individual students. Therefore, any assumptions made about the data should be with extreme 
caution. 
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Figure 4 

Initial SST Request and First SST Meeting 1996-97 



25% 1 
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^ 1996-1997 
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Figure 5 

Initial SST Request and First SST Meeting 1997-98 
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Original Source of SST Referrals 

At the time of the initial referral, data was collected on the referral source for each student. 
Referrals could be initiated in a number of ways, including school staff, outside agencies, or 
parents. Figure 6 displays original referral sources for the 1996-97 and 1997-98 school years 
with percentages rounded. 



Figure 6 

Original Source of SST Referrals 



Source of Request 


1996-97 


1997-98 


Administration 


2% 


2% 


Guidance Counselor 


12% 


3 % 


Nurse 


1% 


0% 


Outside Agency 


1% 


0% 


Parent/ Guardian 


18% 


14% 


Program Specialist 


0% 


N/A 


Psychologist 


0% 


0% 


Teacher 


58% 


75% 


Resource Teacher 


1% 


N/A 


School Social Worker 


2% 


0% 


Special Education Teacher 


3% 


3% 


Central Office 


N/A 


0% 


LART 


N/A 


0% 


SAP 


N/A 


0% 


Other 


2% 


1% 



By a wide margin, regular education teachers (58% in 96-97 and 75% in 97-98) referred 
students to the SST more than any other group, followed by parents (18%, 14%). During the 
1996-97 school year, guidance counselors made 12 percent of the referrals while in the 1997- 
98 school year, they made three percent of the referrals. 

Categories of Presenting Concerns for SST Students 

For the 1997-98 school year, presenting areas of concern were divided into five major 
categories: (1) academic patterns, (2) behavior patterns, (3) emotional/psychological 
behaviors, (4) family characteristics, and (5) other. The categories are summarized in Figure 
7. 
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Figure 7 

Categories of Presenting Concerns for the 1997-98 School Year 



Academic 

Patterns 


Behavior 

Patterns 


Emotional/ 

Psychological 

Behaviors 


Family 

Characteristics 


Other 


Handwriting 


Attention 


Lack of 
Confidence 


Lack of Basic 
Needs 


Attendance 


Math 

Computation 


Aggression 


Inexplicable 

Crying 


Death in a 
Family 


Hearing 


Math Problem 
Solving 


Assignment 

Completion 


Easily Frustrated 


Extended/ 
terminal Illness 


Medical 


Memory 


Following 

Directions 


Exhibits Grief 


Extended mental 
illness of 
immediate 
family member 


Motor Skills 


Oral 

Communication 


Organizational 

Skills 


Mood Changes 


Family changes 
(i.e., separation, 
divorce, new 
member, recent 
move) 


Retention 


Reading 


Social Skills 


Lack of 
Motivation 


Unemployment 


Speech/ 

Language 


Reading 

Comprehension 


Social Skills 


Relationship 

Concerns 


Other 


Vision 


Written 

Language 


Withdrawn 


Low Self-esteem 

Seems 

Preoccupied 




Other 


Other 


Other 


Other 
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Presenting Concerns for SST Students 

For each student referred, major areas of concern were bubbled on the scaimable form. More 
than one concern could be bubbled for each student. The categories for major areas of 
concern were academic, behavior, family, emotional, and other. For the 1996-97 school 
year, academic concerns were the most prevalent (77%). Figure 6 summarizes areas of 
academic concern with percentages rounded. 



Figure 8 

Academic Concerns for 1996-97 



Academic Concern 


1996-97 


Reading 


47% 


Math 


41% 


Written language 


42% 


Recall 


15% 


Oral communication 


9% 


Organization 


25% 


Other Academic concerns 


20% 



During the 1997-98 school year, academic concerns were also the most prevalent concerns 
(84%). For the 1997-98 school year, the categories for academic concerns were coded 
differently. Figure 9 summarizes areas of academic concern with percentages rounded. 

Figure 9 

Academic Concerns for 1997-98 



Academic Concern 


1997-98 


Handwriting 


23% 


Math computation 


36% 


Math problem solving 


43% 


Memory 


26% 


Oral communication 


14% 


Reading 


52% 


Reading Comprehension 


50% 


Written language 


55% 


Other academic concerns 


11% 



For both school years, the second most prevalent concern was behavior (51% for 1996-97, 
81% for 1997-98). Figure 10 summarizes areas of behavior concerns with percentages 
rounded. 
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Figure 10 

Behavior Concerns for 1996-97 



Behavior Concern 


1996-97 


Aggressive 


15% 


Acting out 


27% 


Withdrawn 


9% 


Non-compliant 


22% 


Other behavior concerns 


13% 



Figure 11 summarizes areas of behavior concerns with percentages rounded for the 1997-98 
school year. 

Figure 11 



Behavior Concerns for 1997-98 


Behavior Concern 


1997-98 


Attention 


51% 


Aggression 


18% 


Assignment completion 


47% 


Following directions 


46% 


Organizational skills 


40% 


Social skills 


26% 


Withdrawn 


8% 


Other behavior concerns 


7% 



For both years, the third most prevalent area of concern was emotional (45% for 1996-97, 
65% for 1997-98). Family concerns were reported at 37% (1996-97) and 37% (1997-98). 
For 1996-97, the other category (39%) included AD/HD and medical concerns. For the 
1997-98 school year, the other category (35%) included attendance, hearing, medical, motor 
skills, retention, speech/language, vision, and other. 
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Figure 12 

Presenting Concerns in 1996-97 and 1997-98 




Recommended for Presenting Concerns 

For each student, strategies were recommended by the SST and coded on the initial form. 
Multiple strategies could be recommended. Strategies were grouped differently for the past 
two school years. Figure 13 depicts strategy classifications with percentages rounded. 

Figure 13 

Strategy Recommendations for 1996-97 



Classification 1996-97 



Classroom-based 


79% 


Family support services 


46% 


School counseling 


44% 


School-based support 


39% 


Special education 


31% 


School social work 


31% 


Psychological services 


26% 


Community 


16% 


Health 


13% 



For the 1997-98 school year there were three major strategy classifications; classroom-based 
(87%), school-based (85%), and community-based (16%). Figure 14 depicts the classroom 
strategies and percentage recommended for implementation. Figure 15 depicts the school- 
based strategies and Figure 16 depicts the community-based strategies recommended. 
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Modified instruction, flexible grouping, and parental involvement were the three classroom- 
based strategies recommended most frequently. 

Figure 14 

Percentage of Classroom-based Strategies Recommended in 1997-98 



Strategy 


Percent 

Recommended 


Modified Instruction 
Flexible Grouping 
Parental Involvement 
Modified Environment 
Active Learning Strategies 
Feedback 
Coaching 

Assessment Strategies 
Behavioral Interventions 
Teaching Styles Matched 
Providing Study Skills 
Other Classroom 


52% 
45% 
44% 
40% 
39% 
38% 
34% 
29% 
28% 
21 % 
20% 
10% 



Initial referral to special education, school conference, and individual counseling were the 
three most recommended school-based strategies during the 1997-98 school year. 
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Figure 15 

Percentage of School-based Strategies Recommended in 1997-98 


Strategy 


Percent 

Recommended 


Initial referral to special education 


46% 


School conference 


31% 


Individual counseling 


18% 


Reading specialist 


16% 


Volunteer outside tutor 


13% 


Other school support services 


10% 


Consultation on instructional modification 9% 


Small group counseling 


8% 


Consult with related services 


7% 


Education program planning 


7% 


Home visit 


6% 


Parent education 


6% 


Change in teacher/schedule 


5% 


Change of placement in special education 4% 


Family/community assessment 


4% 


Family counseling 


3% 


Peer mediation 


3% 


Alternative school in WCPSS 


2% 


ESL/Migrant Education 


1% 


Resiliency assessment 


1% 


Medical services, mental health 


services, and community resource referral were the 


community-based strategies most frequently recommended during the 1997-98 school year. 


Figure 16 

Percentage of Community-based Strategies Recommended in 1997-98 


Strategy 


Percent 

Recommended 


Medical services referral 


7% 


Mental health referral 


6% 


Community resource referral 


4% 


Social services referral 


2% 


Alternative school outside WCPSS 


1% 


Legal services referral 


1% 
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Effectiveness of Recommended Strategies 



The strategies recommended for the 1997-98 school year were categorized as classroom 
based strategies, school support services, and community-based services. At the end of the 
year, SST coordinators were asked to code the number of strategies that were recommended 
and the level of effectiveness. 

Multiple strategies could be recommended for a student. As one would expect, not all 
strategies recommended were implemented. For the classroom strategies implemented, the 
strategies were considered fair or good approximately 75 percent of the time. Strategies that 
were used and rated are listed by order of effectiveness in Figure 17. 

Figure 17 

Classroom-based Strategies Used during the 1997-98 School Year 



Effectiveness Rating 


Recommended Strategy 


Good 


Fair 


Poor 


Teaching styles matched (n=514) 


47% 


40% 


13% 


Parental involvement (n=1098) 


45% 


30% 


24% 


Flexible grouping (n=1015) 


42% 


41% 


17% 


Modified instruction (n=l 145) 


42% 


42% 


16% 


Other classroom (n=280) 


42% 


38% 


20% 


Coaching (n=826) 


41% 


42% 


18% 


Assessment strategies (n=689) 


40% 


43% 


17% 


Feedback (n=843) 


40% 


44% 


16% 


Modified environment (n=919) 


39% 


41% 


21% 


Active learning (n=875) 


37% 


46% 


17% 


Providing study skills (n=534) 


34% 


41% 


25% 


Behavior interventions (n=819) 


34% 


42% 


24% 



A number of school support services strategies were recommended for the 1997-98 school 
year. In general, all school support services that were implemented were rated as fair or good 
approximately 75 percent of the time. Strategies that were used and rated are listed by order 
of effectiveness in Figure 18. 
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Figure 18 

School Support-based Services Strategies Used during the 1997-98 School Year 



Effectiveness Rating 



Recommended Strategy 


Good 


Fair 


Poor 


Change of placement in special education (n=215) 


72% 


11% 


16% 


Initial referral to Special Education (n=1041) 


69% 


23% 


8% 


Peer mediation (n=202) 


62% 


28% 


10% 


Consult with related services (n=232) 


57% 


24% 


19% 


Individual counseling (n=874) 


57% 


34% 


10% 


Alternative School in WCPSS (n=139) 


56% 


35% 


8% 


Small group counseling (n=376) 


56% 


35% 


9% 


Reading specialist (n=407) 


56% 


36% 


8% 


Volimteer outside tutor (n=412) 


52% 


34% 


15% 


Change in teacher/schedule (n=235) 


51% 


32 


17% 


Consultation on instructional modification (n=288) 


51% 


39% 


10% 


Education program planning (n=234) 


50% 


34% 


16% 


School conference (n=942) 


50% 


34% 


17% 


Parent education (n=240) 


47% 


33% 


20% 


Family coimseling (n=195) 


47% 


34% 


19% 


Home visit (n=219) 


45% 


31% 


24% 


Other (n=243) 


42% 


43% 


15% 


ESL/Migrant education (n=95) 


39% 


44% 


17% 


Resiliency assessment (n=l 17) 


37% 


53% 


11% 


Family/community assessment (n=177) 


35% 


44% 


21% 



Community-based services were the third category of recommended strategies for the 1997- 
98 school year. With the exception of legal services referrals, most comm un ity-based 
strategies were rated good or fair approximately 75 percent of the time. Figure 19 depicts 
community-based strategies in order of effectiveness. 

Figure 19 

Community-based Services Strategies Used during the 1997-98 School Year 





Effectiveness Rating 


Recommended Strategy 


Good 


Fair 


Poor 


Medical services (n=272) 


65% 


24% 


12% 


Alternative school outside WCPSS (n=143) 


65% 


26% 


10% 


Community resource (n=206) 


57% 


30% 


13% 


Social services (n=145) 


46% 


29% 


25% 


Mental health (n=257) 


46% 


38% 


16% 


Legal services (n=88) 


46% 


14% 


41% 
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Improvement for Presenting Concerns 

For each student served by the SST, a follow-up scannable form was completed towards the 
end of the year. For each initial presenting concern, the team member completing the form 
was to indicate whether the concern had improved a lot, improved somewhat, or had not 
improved. The concerns categories were academic performance, behavior, dealing with 
family problems, emotional, and other concerns. Figure 20 summarizes the data for 
improvement of presenting concerns for the 1996-97 school year. 

Figure 20 

Improvement for Presenting Concerns in 1996-97 



60% ^ 




Academic Concern Behavior Concern Family Concern Emotional Concern Other Concern 

(n=544) (n=356) (n=241 ) (n=281 ) (n=245) 



Concerns 

Himproved ^Somewhat Improved *No Improvement 



The same procedure was used for the 1997-98 school year. A scannable form was completed 
at the time of referral and towards the end of the school year. The presenting concern 
categories were academic patterns, behavior patterns, emotional/psychological behaviors, 
family characteristics, and other. (Descriptors for each category are provided in Figure 7 on 
page 10.) Each concern was rated as had improved a lot, improved somewhat, or had not 
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improved. Figure 21 summarizes the data for improvement of presenting concerns for the 
1997-98 school year. 



Figure 21 

Improvement for Presenting Concerns in 1997-98 




Academic Concern Behavior Concern 

n=1521 na1557 



Family Concern Emotional Concern Other Concern 

n=:11267 n=868 ns186 

Concerns 



l^^lmproved ^Somewhat Improved Mnq improvement 



Presenting concerns for kindergarten through second grade were categorized as academic, 
behavior, emotional, family, or other concerns. For all categories, some improvement was 
reported. Figure 22 summarizes the data for improvement of presenting concerns for the 
1997-98 school year with percentages rounded. 



Figure 22 

Improvement of Presenting Concerns for Grades K-2 in 1997-98 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


13% 


52% 


65% 


35% 


Behavior 


17% 


46% 


64% 


37% 


Emotional 


10% 


51% 


61% 


39% 


Family 


9% 


43% 


52% 


48% 


Other 


11% 


42% 


53% 


47% 
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The same categories of presenting concerns used for kindergarten through second grades 
were used for grades three through five also. There was improvement for each area of 
concern. However, for family concerns (n=299), approximately 52 percent reported no 
improvement compared to approximately 48 percent improvement. Figure 23 summarizes 
the data for improvement of presenting concerns for the 1997-98 school year. 

Figure 23 

Improvement of Presenting Concerns for Grades 3-5 1997-98 School Year 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


15% 


50% 


65% 


35% 


Behavior 


15% 


49% 


65% 


36% 


Emotional 


12% 


50% 


62% 


38% 


Family 


9% 


39% 


48% 


52% 


Other 


14% 


38% 


53% 


47% 



In grades six through eight, firom the time of initial concern to the end-of-the year, all 
concerns improved some. Improvement for presenting concerns ranged firom approximately 
45 percent to 56 percent. However, for emotional concerns and family concerns the percent 
of “no improvement” (53 percent, 56 percent) was higher than the percent of improvement. 
Figure 24 summarizes the data for improvement of presenting concerns for the 1997-98 
school year for six through eighth grades. 



Figure 24 

Improvement of Presenting Concerns for Grades 6-8 1997-98 School Year 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


20% 


37% 


56% 


44% 


Behavior 


16% 


38% 


54% 


46% 


Emotional 


16% 


31% 


47% 


53% 


Family 


10% 


35% 


45% 


56% 


Other 


19% 


33% 


51% 


49% 



All areas of presenting concerns showed improvement for high school students by the end of 
the year. The ranges for total improvement were approximately 74 percent (other) to 84 
percent (emotional). In general, the percent of improvement for high school students was 
greater than for students at other grade levels. Total improvement for academic (n=220) was 
approximately 77 percent, behavior (n=390) was 79 percent, emotional (n=283) was 84 
percent, family (n=380) was approximately 75 percent, and other (n=237) was approximately 
74 percent. 
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Figure 25 

Improvement of Presenting Concerns for High School Students in 1997-98 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


20% 


58% 


77% 


23% 


Behavior 


27% 


53% 


79% 


21% 


Emotional 


25% 


59% 


84% 


16% 


Family 


22% 


53% 


75% 


25% 


Other 


20% 


54% 


74% 


26% 



Comparison of Initial Referral Grades with End of Year Grades 

SST coordinators provided information on the initial scan form when a student was referred 
to the SST. In addition to recording presenting concerns, the student’s current grades were 
also recorded. Grades recorded on the initial scan form were compared with student grades 
recorded on the end-of-year scan form. At the end of the year, SST coordinators were also 
asked to provide information related to improvement of initial referral concerns. The 
following information is provided for students in kindergarten through grade five. Only 
those students whose initial referral and end-of-year scan sheets were returned are included. 

Comparison of Initial Referral Grades with End of Year Grades for 
Kindergarten Students 



Figure 26 

Kindergarten Grades for the 1996-97 School Year 



Skills 


Improved 


Stayed the Same 


Worsened 


Social Skills 


34% 


51% 


14% 


Work Habits 


27% 


53% 


21% 


Language Skills 
Total n=34 


32% 


38% 


29% 
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Figure 27 

Kindergarten Grades for the 1997-98 School Year 



Skills 


Improved 


Stayed the Same 


Worsened 


Social Skills (n=89) 


34% 


36% 


30% 


Work Habits (n=89) 


36% 


28% 


36% 


Language Skills (n=87) 


28% 


41% 


31% 



Comparison of Initial Referral Grades with End of Year Grades for 
Grades 1 & 2 



The same procedure for grade collection was followed for first and second grade students 
who were referred to the SST. Grades recorded on the initial scan form were compared with 
student grades recorded on the end-of-year scan form. SST coordinators filled in information 
on the initial scan form when a student was initially referred to the SST. They recorded final 
grades at the end of the school year. 

For the 1996-97 school year, students were graded on reading, written language, math 
computation, math problem solving, completes assignments, and follows directions. 
Commendable, satisfactory, or needs improvement were the grades assigned. In addition, an 
instructional level (above, on, below grade level) was assigned for reading, written language, 
math computation, and math problem solving. For all categories, most students either 
improved or stayed the same. 



Figure 28 

First and Second Graders End-of-Year Grades in 1996-97 



Skills 


Improved 


Stayed the Same 


Worsened 


Reading (n=91) 


23% 


71% 


5% 


Written Language (n=92) 


24% 


66% 


10% 


Math Computation (n=92) 


18% 


71% 


11 % 


Math Problem Solving (n=90) 


18% 


70% 


12% 


Completes Assignments (n=87) 


29% 


55% 


16% 


Follows Directions (n=88) 


23% 


58% 


19% 



A similar pattern was shown in grade improvement for the 1997-98 school year. Most 
students either improved or stayed the same compared with the percentage that worsened. 
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Figure 29 

First and Second Graders End-of-Year Grades in 1997-98 



Skills 


Improved 


Stayed the Same 


Worsened 


Reading (n=480) 


24% 


66% 


11% 


Written Language (n=477) 


20% 


69% 


11% 


Math Computation (n=467) 


19% 


69% 


12% 


Math Problem Solving (n=464) 


16% 


70% 


14% 


Completes Assignments (n=453) 


28% 


61% 


11% 


Follows Directions (n=443) 


25% 


63% 


13% 



Comparison of Initial Referral Grades with End of Year Grades for 
Grades Three through Five 

Grades were recorded at the time of initial referral and then at the end of the school year. For 
each academic area during the 1996-97 school year, approximately 20 to 29 percent 
improvement was shown. Language arts and mathematics had the highest percent 
improvement. For approximately 39 to 49 percent, grades remained the same. For 
approximately 23 to 39 percent, grades worsened. 

Figure 30 

Third through Fifth Graders End-of-Year Grades in 1996-97 



Skills 


Improved 


Stayed the Same 


Worsened 


Language Arts (n=73) 


29% 


44% 


27% 


Mathematics (n=74) 


28% 


49% 


23% 


Science(n=71) 


21% 


39% 


39% 


Social Studies (n=70) 


23% 


44% 


33% 



For the 1997-98 school year, third through fifth graders showed improved grades for all 
academic areas. The range of improvement was somewhat higher than for 1996-97. The 
range for 1997-98 was approximately 29 percent to 33 percent compares to 20 to 29 for the 
previous year. 



Figure 31 

Third through Fifth Graders End-of-Year Grades in 1997-98 



Skills 


Improved 


Stayed the Same 


Worsened 


Language Arts (n=43 1 ) 


30% 


53% 


17% 


Mathematics (n=426) 


29% 


52% 


20% 


Science(n=402) 


29% 


45% 


26% 


Social Studies (n=410) 


33% 


42% 


24% 
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EVALUATION QUESTIONS 

The purpose of this evaluation was to respond to several questions related to the types of 
student referrals, the types of strategies recommended and implemented, and the 
effectiveness of the SST program. 

Did the number of meetings and referrals occur at the same rate during the year? 

From the data collected for the 1997-98 school year, it appears that meetings and referrals 
occurred at about the same rate for most of the year. There was an initial lag during August 
and September, when referrals exceeded initial meetings. This pattern was also true for the 
1996-97 school year. 



Were a large majority of the students identified as exceptional learners? 

No. Most students were not identified as exceptional learners. In addition, few of the 
students (two-percent) were receiving services in an ESL program. 



Were presenting concerns more prevalent in any particular area(s)? 

Academic concerns followed by behavior were the most prevalent during the 1997-98 school 
year. This was also true of the 1996-97 school year. 



Did students’ academic grades improve after SST interventions? 

Approximately 66 percent of academic concerns were reported as “somewhat” or “a lot” 
improved for the 1997-98 school year. Actual grades varied for elementary students. For the 
1997-98 school year, approximately a third of the kindergarten grades improved in social 
skills and work habits. Language skills improved for approximately 28 percent of students. 
The percentage of students who improved in first and second grades ranged from 
approximately 16 percent for math problem solving to 28 percent for completing assignments 
during the 1997-98 school year. Improvement for third through fifth graders ranged from 
approximately 29 percent in mathematics and science to 30 percent in language arts. It 
should be noted that most grades remained the same for all elementary students during the 
1997-98 school year. 

Did students who were referred for behavior show improvement after SST interventions? 

Yes. The majority of students with behavior concerns showed some improvement. Figure 
32 depicts the total percentage of improvement. 
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Figure 32 

Improvement of Behavior Presenting Concerns in 1997-98 



Grade Level 


Total Improved 


No Improvement 


K-2 


64% 


37% 


3-5 


65% 


36% 


6-8 


54% 


46% 


High School 


79% 


21% 



Did students who were referred for emotional stability concerns show improvement after 
SST interventions? 

With the exception of middle school, most students showed improvement. . For middle school 
students, 47 percent showed improvement while 53 percent showed no improvement for 
emotional presenting concerns. If this trend continues in the middle school, further 
investigation will be needed to determine why more students are not showing improvement. 

Figure 33 

Improvement of Emotional Presenting Concerns in 1997-98 



Grade Level 


Total Improved 


No Improvement 


K-2 


61% 


39% 


3-5 


62% 


38% 


6-8 


47% 


53% 


High School 


84% 


16% 
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Did students who were referred for “other” concerns show improvement after SST 
interventions? 

Yes, there was some improvement shown for all grade levels during the 1997-98 school year. 
For kindergarten through eighth grade, the total improvement and no improvement were 
fairly close. For high school, total improvement was significantly higher than for those who 
did not improve. It should be noted that the “other” category is quite varied. For example, it 
includes an attendance category and an “other” category. 

Figure 34 

Improvement of Other Presenting Concerns in 1997-98 



Grade Level 


Total Improved 


No Improvement 


K-2 


53% 


47% 


3-5 


53% 


47% 


6-8 


51% 


49% 


High School 


74% 


26% 



Were some strategies more effective than others? 

Strategies for the 1997-98 school year were categorized as classroom-based (see Figure 14, 
page 14), school support strategies (see Figure 15, page 15) or community-based (see Figure 
16, page 15). For each individual strategy that was recommended and implemented, the 
percentage that reported an effectiveness rating of “good” was much higher than those rated 
as “poor.” This might suggest that the types of strategies that were recommended and 
implemented were quite appropriate and successful for those students. 

It appears that the SST program continues to provide an invaluable service to students at-risk 
and their families. Meetings for student referrals were held in a time sensitive manner. For 
those students referred, a large majority showed improvement for their areas of presenting 
concerns by the end of the year. In addition, those recommended and implemented strategies 
were highly successful for classroom-based, school support services, and community-based 
services. 
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RECOMMENDATIONS 

The SST program provides an important service to students who are at-risk and their 
families. The implementation of technology to assist in program evaluation has proven 
effective. In the future, it may be helpful to collect additional data using the scannable 
forms. For example, SSTs may find it helpful to know whether an individual student is 
referred multiple times during a school year. They may also wish to code when a student 
leaves the system prior to the end of the school year. SSTs may also wish to know the length 
of time between strategy recommendations and implementation, as well as intensity of the 
actual implementation. 

Quantitative data provides an invaluable resource in program evaluation. However, it does 
not always provide the full picture. A qualitative component to the intervention/prevention 
evaluation could provide invaluable information for program improvement. It should be 
noted that qualitative methods often require more resources for data collection and more time 
for analysis than quantitative data. There are a few software programs that may be useful in 
analyzing qualitative data. 

The Intervention/Prevention staff and the Evaluation and Research staff should meet in the 
spring to determine what evaluation questions should be answered in the following school 
year. At that time, staff could deter mi ne whether a qualitative component is desired and 
feasible for future evaluation efforts. 



J3 
o 

ERIC 



27 



INTERVENTION/PREVENTION GRANT EVALUATION 1996-98 



Authors 

Doris Tyler 

1998-99 Evaluation Specialist 

Deborah Manzo 
1996-98 Evaluation Specialist 



Contributing Staff 

Chuck Dulaney 
Director of Evaluation 

Alissa Bemholc 
Data Analyst 

Lisa Cassidy 
Graduate Intern 

Linda Hess 
Evaluation Secretary 



January 1999 
E&R Report No. 99.16 



Department of Evaluation and Research 
Wake County Public School System 



$ 




w 

U.S. Department of Education 
Office of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 







TM029931 



I. DOCUMENT IDENTIFICATION: 





L-n+erveMuin/PreVen-Viori Grarrb Ev£!.luci+ibo 


Author(s):'T?oris~l~vl£r . "IDeborcth 


^ 

Corporate Source: | 

Cour>~l-^ PtAiolLe SoVobb ^ ^ 1 


1 Publication Date: 



II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced 
in the monthly abstract journal of the ERIC system. Resources in Education (RIE), are usually made available to users In microfiche, reproduced 
paper copy, and electronic/oplicaJ media, and sold through the ERIC Document Reproduction Service (EDRS) or other ERIC vendors. Credit Is 
given to the source of'each document, and. if reproduction release is granted, one of the following notices is affixed to the document 

If permission Is granted to reproduce and disseminate the identif'ied document, please CHECK ONE of the fol towing two options and sign at 
the bottom of the page. 



t 

Check here 
For Level 1 Release: 

Permitting reproduction in 
microfiche (4" x 6* film) or 
other ERIC archival media 
(e.g., electronic or optical) 
and paper copy. 



The sample sticker shown below will be The sample sticker shown below will be 
affixed to all Level 1 documents to Level 2 documents 

□ 

Check here 
For Level 2 Release: 

Pennitting reproduction in 
microfiche (4* x 6" film) or 
other ERIC archival media 
(e.g., electronic or optical), 
but not in paper copy. 



PERMISSION TO REPRODUCE AND 




PERMISSION TO REPRODUCE AND 


DISSEMINATE THIS MATERIAL 




DISSEMINATE THIS 


HAS BEEN GRANTED BY 




MATERIAL IN OTHER THAN PAPER 






COPY HAS BEEN GRANTED BY 





















V 






TO THE EDUCATIONAL RESOURCES 




TO THE EDUCATIONAL RESOURCES 


INFORMATION CENTER (ERIC) 




INFORMATION CENTER (ERIC) 



Level 1 



Level 2 



Documents will be processed as indicated provided reproduction quality permits. If permission 
to reproduce is granted, but rteither box is checked, documents will be processed at Level 1 . 



Sign 

here-* 

please 




Vhareby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate 
this document as indicated above. Reproduction from the ERIC microfiche or eiectronic/optical media by persons other than 
ERIC employees and its system contractors requires permission from too copyright holder. Exception is made for mn profit^ 
reproduction by l^raries and oto^ service agencies to satisfy information needs of educators in response to discrete inquiries. 


SignaturaiO 


1 Printed Name/Position/Tide: 


/ Ei/aluaiio'n Sc - 


Org^aation/Address: _ 

LOciWe. CourH^Public oakools 


X T^epHone: 


1 

1 


iFAX: 

1 <?/'?') ^b~IShl 


. /UC 


i^Mail Address: 

1 

t 

! 


I 


1 Date: ' 

1 




